A Holistic Approach to the Evaluation of Data Warehouse Maintenance Policies

The research community is addressing a number of issues in response to increased reliance of organisations on data warehousing. Most work addresses individual aspects related to incremental view maintenance, propagation algorithms, consistency requirements, performance of OLAP queries etc. There remains a need to consolidate relevant results into a cohesive framework for data warehouse maintenance. Although data propagation policies, source database characteristics, and user requirements have been addressed individually, their co-dependencies and relationships have not been explored. In this paper, we present a comprehensive, cost-based framework for evaluating data propagation policies against data warehouse requirements and source database characteristics. We formalize data warehouse specification along the dimensions of freshness (or staleness), response time, storage, and computation cost, and classify source databases according to their data propagation capabilities. A detailed cost model is presented for a representative set of policies. A prototype implementation has allowed an exploration of the various trade-offs. The results presented in this paper are for a single source, but the approach and the framework are extensible. Current work is addressing a broader class of sources and a more detailed data warehouse specification that includes multiple sources.